sqlite: typed function/operator surface, unified with the PG codegen#82
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR overhauls the SQLite typed-function surface generation: it replaces the prior runtime-only inference approach with a doc-derived, mechanically verified signatures pipeline, then emits placed methods/free functions with strict @expose runtime argument validation for exoeval/RPC safety.
Changes:
- Add a doc-backed SQLite signature inventory (
signatures.ts+docs/*.json) and a deterministic emitter (emit.ts) that generates the SQLite method/function surface. - Add a property-based verifier that checks every signature claim against the actual SQLite engine (
signatures.verify.test.ts) and broaden SQLite smoke tests to cover cross-surface composition and runtime gating. - Adjust SQLite runtime behavior and surrounding plumbing (new
Anyroot, boolean/int binding normalization, unary-op support, and codegen script updates).
Reviewed changes
Copilot reviewed 31 out of 41 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| vitest.config.ts | Exclude .claude/ from Vitest scanning. |
| src/types/sqlite/smoke.test.ts | Expands smoke tests to cover generated surface breadth, placement, and runtime schema gating. |
| src/types/sqlite/signatures.verify.test.ts | Adds engine-backed verifier for doc-extracted signatures + completeness/determinism checks. |
| src/types/sqlite/signatures.ts | Loads and validates per-doc-page JSON signature facts + defines EXCLUSIONS policy list. |
| src/types/sqlite/overrides/real.ts | Removes hand-written Real override in favor of generated view class. |
| src/types/sqlite/overrides/integer.ts | Removes hand-written Integer override in favor of generated view class. |
| src/types/sqlite/overrides/bool.ts | Reworks Bool to extend generated bool-subject methods and keep hand-written connectives/statics. |
| src/types/sqlite/overrides/any.ts | Introduces hand-written SQLite root Any over generated universal surface, including bespoke .in(). |
| src/types/sqlite/index.ts | Updates SQLite barrel exports (root Any, generated views, and generated free functions). |
| src/types/sqlite/generated/support.ts | Adds shared generation helpers (zod schema builder + numeric result strategy). |
| src/types/sqlite/generated/bool.ts | Adds generated bool-subject method base class (extended by hand-written Bool). |
| src/types/sqlite/generated/blob.ts | Regenerates Blob view class and placed blob-subject methods with strict @expose schemas. |
| src/types/sqlite/generated/any.ts | Adds generated universal-method base class (extended by hand-written Any). |
| src/types/sqlite/generate.ts | Removes legacy runtime-only SQLite inference/emitter script. |
| src/types/sqlite/emit.ts | Adds deterministic emitter that places methods by subject domain and emits free functions + barrel updates. |
| src/types/sqlite/docs/VERSION | Records doc snapshot stamp and target SQLite version. |
| src/types/sqlite/docs/percentile.json | Adds extracted signature facts for percentile extension aggregates. |
| src/types/sqlite/docs/lang_mathfunc.json | Adds extracted signature facts for math functions. |
| src/types/sqlite/docs/lang_expr.json | Adds extracted signature facts for expression operators. |
| src/types/sqlite/docs/lang_datefunc.json | Adds extracted signature facts for datetime functions/modifiers and subject placement. |
| src/types/sqlite/docs/lang_corefunc.json | Adds extracted signature facts for core functions (incl. deterministic vs non-deterministic). |
| src/types/sqlite/docs/lang_aggfunc.json | Adds extracted signature facts for aggregates. |
| src/types/sqlite/docs/json1.json | Adds extracted signature facts for JSON functions/operators. |
| src/types/sqlite/docs/index.ts | Defines the doc-extraction schema/contract and extraction prompt used to produce docs/*.json. |
| src/types/sqlite/docs/fetch.sh | Adds script to vendor SQLite doc pages and stamp VERSION. |
| src/types/sqlite/docs/.gitignore | Ignores vendored .html doc pages (JSON artifacts remain committed). |
| src/types/sqlite/base.ts | Removes legacy SqliteValue base class (replaced by SQLite Any). |
| src/types/runtime.ts | Adds UnaryOp support for dialect-tagged unary operator AST construction. |
| src/tables/sqlite.ts | Updates affinity resolution to map NUMERIC fallthrough to Any (new SQLite root). |
| src/tables/sqlite.test.ts | Updates tests and generated-code snapshot expectations for Any vs SqliteValue. |
| src/tables/generate.ts | Updates documentation/comments for resolved SQLite type class naming. |
| src/exoeval/tool.ts | Adds expose.rest(...) / RestSchema support for variadic argument validation. |
| src/driver.ts | Normalizes better-sqlite3 bindings: integers as BigInt and booleans as 0n/1n for correct storage/semantics. |
| src/builder/query.ts | Updates comments to reflect SQLite Any as the common SqlValue-based dialect root. |
| package.json | Switches SQLite codegen scripts from generate.ts to emit.ts. |
| .gitignore | Ignores .claude/. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+51
to
+55
| in(...vals: [Any<any> | boolean | number | string, ...(Any<any> | boolean | number | string)[]]): types.Bool<any> { | ||
| const wrapped = vals.map((v) => { | ||
| if (v instanceof Any) {return v;} | ||
| if (!isPlainData(v)) { | ||
| const name = (Object.getPrototypeOf(v) as { constructor?: { name?: string } } | null)?.constructor?.name ?? "anonymous"; |
|
|
||
| const HEADER = "// Auto-generated by src/types/sqlite/emit.ts from signatures.ts — do not edit.\n"; | ||
|
|
||
| const CLS: Record<SqlT, string> = { |
| const CLS: Record<SqlT, string> = { | ||
| integer: "Integer", real: "Real", text: "Text", blob: "Blob", bool: "Bool", any: "Any", | ||
| }; | ||
| const PRIM: Record<SqlT, string> = { |
Comment on lines
+1
to
+4
| // BLOB storage-class view — hand-written behavior over the generated | ||
| // method surface (../generated/blob.ts): Uint8Array instances are the | ||
| // accepted primitive (instanceof, not typeof), and the driver hands | ||
| // blobs back as Uint8Array — the declare only narrows the return type. |
Comment on lines
+7
to
+10
| export class Blob<in out N extends number> extends Generated<N> { | ||
| static override acceptsPrimitive(v: unknown): boolean { return v instanceof Uint8Array; } | ||
| declare deserialize: (raw: string) => Uint8Array; | ||
| } |
Comment on lines
170
to
172
| throw new Error( | ||
| `${this.__typnameText}.serialize: expected a ${this.__typnameText} instance or ${expected} primitive, got ${typeof v} (${String(v).slice(0, 60)}).`, | ||
| `${this.__typnameText}.serialize: expected a ${this.__typnameText} instance or ${this.primitiveTs} primitive, got ${typeof v} (${String(v).slice(0, 60)}).`, | ||
| ); |
Both pg and sqlite feed one emitter a typed schema of facts describing their function catalog. pg's facts are auto-generated on the fly from pg_catalog (as before — generated output is byte-identical). sqlite's facts are generated by running an LLM over the sqlite docs pages, committed as JSON, and sanity-checked against the real engine with deterministic quick-check-style tests.
ryanrasti
force-pushed
the
ryan_sqlite_fns
branch
from
ba51146 to
0cf5695
Compare
ryanrasti
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.